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METHOD AND DEVICE FOR DRIVER INFORMATION AND FOR A REACTION 

WHEN LEAVING THE TRAFFIC LANE 

FIELD OF THE INVENTION 

The present invention relates to a method and a device for 
driver information and a reaction upon departure from the 
lane, or, more accurately, in response to the threatening 
leaving of the lane . 

BACKGROUND INFORMATION 

Warning systems, which warn the driver when he leaves the 
lane, or rather, threatens to leave the lane, are known. For 
example, European Patent Application No. 1 074 43 0 discusses a 
method in which, based on image sensor systems, the lane in 
which the vehicle is moving is ascertained, and the driver is 
warned when the vehicle leaves the lane or threatens to do so. 
However, information on the exact boundary conditions, under 
which such a warning is given, is not stated. 

SUMMARY OF THE INVENTION 

By the driver's taking into account future corrections of the 
track (path) of the vehicle, in ascertaining whether, due to a 
future departure from the lane, he should be warned or not, it 
is possible effectively to avoid false warnings and, 
nevertheless, to maintain the reliability of the warning in 
response to the occurrence of the corresponding traffic 
situation. 

This may allow recognition of a future departure from the lane 
in an exact and reliable manner. This fact and the suppression 
of false warnings, increases the operating friendliness, and 
the acceptance of a driver assistance system in which the 
driver is warned when the vehicle leaves the lane or upon 
future departure from the lane, or rather, reactions are 
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introduced which lead the vehicle back to its normal lane 
(such as by a steering intervention) . 



In response to the estimation of the future path of the 
5 vehicle, future corrections of the path by the driver may be 
taken into consideration in an execution that includes his 
reaction time- This may lead to further improvement of 
accuracy and reliability of the system. 

10 An average value of the correction measures of the driver, 
such as a steering angle value or a transverse acceleration 
value is predefined, and may be taken into consideration in 
determining the path. The state of attention of the driver may 
be taken into consideration in forming the magnitude of such a 

15 predefined value. 

In the calculation of the future path of the vehicle, which is 
drawn upon to warn the driver when he leaves the lane, the 
state of attention of the driver may be taken into account in 

20 such a manner that the behavior of the driver in the past is 
ascertained, and, in response to behavior which leads one to 
conclude that the driver is inattentive, the possible future 
path of the vehicle may be calculated using a smaller change 
in the curvature than for a driver that is presumed to be 

25 attentive. The steering motions or the yaw rate changes in the 
past may be drawn upon for estimating the degree of 
attentiveness of the driver. Small steering motions designate 
an inattentive driver, and more forceful motions an attentive 
one . 

30 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows a block diagram of a device for warning a 
driver and for a reaction when the vehicle threatens to leave 
the lane. 

35 

Figure 2 shows a sketch of a flow chart in which a first 
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method for determining the driver warning is shown. 



Figure 3a shows a sketch of a traffic situation . 

5 Figure 3b shows a sketch of a traffic situation. 

Figure 4 shows a flow chart for a second exemplary embodiment. 

Figure 5 shows a traffic situation in light of the method of 
10 Figure 4. 

Figure 6 shows an exemplary embodiment in which the 
calculation of the possible future path of the vehicle is a 
function of the degree of attentiveness of the driver and 
15 shows the situation for an attentive driver. 

Figure 7 shows an exemplary embodiment in which the 
calculation of the possible future path of the vehicle is a 
function of the degree of attentiveness of the driver and 
2 0 shows the situation for an inattentive driver. 

DETAILED DESCRIPTION 

Figure 1 shows a device used for warning a driver and for a 
reaction in response to future leaving of the lane of the 

25 vehicle. What is shown is a control or evaluation unit 10, 

which includes an input circuit 12, a microcomputer 14 and an 
output circuit 16. These elements are interconnected via a bus 
system for mutual data exchange. Input lines from various 
measuring devices, via which measuring signals and measuring 

30 data are transmitted, are supplied to input circuit 12. A 
first input line 2 0 connects input circuit 12 to an image 
sensor system 22, which picks up the scene in front of the 
vehicle. Corresponding image data are transmitted via input 
line 20. Furthermore, input lines 24 to 28 are provided which 

35 connect input circuit 12 to measuring devices 3 0 to 34. These 
measuring devices are, for example, measuring devices for 
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measuring the vehicle speed, for recording the steering angle 
and for recording additional operating variables of the 
vehicle, which have meaning in connection with the method 
described below. At least one warning device 3 8 is activated 
5 via output circuit 16 and output line 36, for instance, a 

warning lamp and/or a loudspeaker for an acoustical warning or 
a voice response and/or a display for displaying an image, 
with the aid of which the driver is informed and/or warned 
before the departure, to be feared, from the lane, according 

10 to the method shown below. In addition, or alternatively, it 
is provided in some exemplary embodiments that one should 
activate a controlling system 42 via output circuit 16 and an 
output line 40, which automatically leads back into the lane, 
for instance by an intervention in the steering of the 

15 vehicle, and thus prevents leaving the lane. 

According to the method described below, a warning or an 
intervention in the vehicle movement occurs when it is 
estimated that the driver is no longer able to hold the 

20 vehicle in the lane using normal corrections. To do this, a 

check is made whether, while considering normal corrections of 
the driver, i.e. a certain steering angle which the driver 
normally uses to correct his track, the vehicle crosses the 
edge markings or not. The future track of the vehicle is then 

25 specifically calculated while taking into consideration these 
corrections by the driver that are to be expected, is compared 
to the detected edge markings, and a warning is emitted if the 
future track of the vehicle crosses the edge markings. In the 
determination of the value represented by the correction of 

3 0 the track by the driver that is to be expected, in one 

supplementary exemplary embodiment, the state of attentiveness 
of the driver is taken into consideration. In this context, 
the value is the greater (this corresponds to a more forceful 
correction) , the more attentive the driver is, and the less, 

35 the more inattentive the driver is. In this context, the 
attentiveness of the driver is derived, depending on the 
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exemplary embodiment, from steering wheel movements, head 
posture, accelerator motion, etc. 

In an additional exemplary embodiment, the reaction time of 
5 the driver is also estimated in connection with the detection 
of whether the vehicle will leave the lane in the future. It 
is assumed that, at the beginning of a warning, the driver 
will have a certain reaction. It has proven suitable to 
subdivide this reaction into several phases. In a first phase 

10 (of a duration such as 0.5 sec) no change in the steering 

angle occurs (reaction time of the driver to the warning) ; in 
a second phase, which borders on the first phase (of a 
duration such as 0.1 sec), the driver undertakes a correction 
away from the edge marking. He may, for example, turn the 

15 steering wheel by a certain angle (e.g. 10°) in the direction 
that brings the vehicle back into the lane. Alternatively to 
this, a rotation of the wheel is performed so that a curve is 
driven that has a transverse acceleration of 2 m/sec^. In a 
third phase, which borders on the second phase, driving 

2 0 continues at an unchanged steering angle, until the vehicle 
moves in the direction of the center of the lane. Under 
consideration this behavior pattern, the future track for each 
point in time, which the vehicle will travel in case of a 
warning, is calculated. From this variable, the suitable point 

25 in time of the warning may then be derived, which has to occur 
exactly when the track, calculated in consideration of the 
driver's reaction, crosses the corresponding lane marking by a 
desired measure (which may also be negative) . The range of 
distance in which the comparison with the edge marking occurs 

30 is limited, in this context (for instance, to 1 sec) . These 

measures ensure that, in each lane- leaving situation, after a 
warning, the driver gets away from his own lane at most only 
by a certain measure. 

35 Alternatively, or in supplement of the warning, an 

intervention in the vehicle movement occurs, e.g. in the 
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steering, by which the vehicle gets back to the track again in 
a lane- leaving situation as described above. 

The above -described methods are implemented using programs 
5 which are programmed on microcomputer 14 • The latter generates 
the output signals for the warning or vehicle intervention, 
with the aid of the programs and according to the input 
signals supplied. In the light of flow diagrams, two exemplary 
embodiments of the method represented above are shown below. 

10 

A first exemplary embodiment is described in the light of the 
flow chart of Figure 2. In this context, the program 
represented is run through at certain points in time. 

15 In first step 100, the sequence of the lane marking (such as 
left and/or right lane edge) is read in. The sequence of the 
lane marking is given, in this context, in one exemplary 
embodiment as a polynomial, such as a third order power 
function. This function is derived from the image data 

2 0 ascertained by the image sensor system In this context, the 

microcomputer, using object recognition measures, recognizes 
the markings, and calculates the parameters of the power 
function from the recognized markings. In subsequent step 102 
the future track of the vehicle is then calculated. The 
25 representation of this track also occurs using the function 
type that is used for the sequence of the lane markings . 
Accordingly, for example, from the current vehicle speed, the 
current steering angle or the transverse acceleration, etc, 
the parameters of the function are calculated which calculate 

3 0 the pattern of at least one track (track of the left and/or 

right vehicle edge) of the vehicle. An additional variable, 
that is evaluated for the calculation of the parameters, is 
the predefined correction value that may be a function of the 
attentiveness state, which is worked in, for example, by the 
35 correction of the steering angle. In subsequent step 104 the 

two functions are then checked as to whether they intersect or 
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whether the future vehicle track lies across the pattern of 
the edge markings beyond a certain measure, or whether the 
future vehicle track is approaching the edge markings up to a 
certain measure. The distance range (distance from the 
5 vehicle) , to which this comparison is limited, is predefined, 
in this context, (e.g. 1 sec), or is predefined by the maximum 
measuring distance of the image sensor system. If no crossing 
(point of intersection) is ascertained, one must assume that 
the vehicle remains in the lane, and the program is ended. If, 

10 however, a crossing of the two track curves are ascertained, 
then, according to step 106 a warning to the driver occurs, 
using acoustical and/or optical arrangement, and in one 
exemplary embodiment an automatic correction of the track of 
the vehicle, for example, by an intervention in the steering 

15 of the vehicle. The program is then ended and run through 
again at the next point in time. 

Figure 3 shows two traffic situations, according to Figure 3a, 
no driver warning occurs, and according to Figure 3b, a driver 

20 warning occurs. Future lane 202 calculated for vehicle 200, 
which is determined, as shown above, taking into account 
possible corrections by the driver (e.g. fixed steering angle 
value) , in Figure 3a shows no crossing with the edge marking 
shown as the dashed line, whereas in Figure 3b such a crossing 

25 is present. According to that, in Figure 3b the driver is 
warned at the time shown or a vehicle intervention is 
undertaken, since the driver will probably not hold the 
vehicle in the lane only by normal corrections, while in 
Figure 3a the warning does not occur because the driver will 

3 0 be able to do so. 

The extent of the assumed correcting motion of the driver may 
be a function of his state of attentiveness . In the case of a 
watchful driver, the correcting movement to be expected will 
35 lie, for example, at a steering angle of 10° and a transverse 
acceleration of 2 m/sec^, whereas this value is clearly lower 
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in the case of an inattentive driver. The state of 
attentiveness of the driver is derived, for example, from 
steering movements, accelerator movements, body posture, etc, 
and then, as a function of the degree of attentiveness, the 
5 correcting values are determined, and these are incorporated 
in the calculation of the future track of the vehicle. 

A second exemplary embodiment, which further improves the 
accuracy and reliability of the recognition of the existing 

10 situation, is shown in the flow diagram in Figure 4. This 
program, too, is run through at predefined time intervals. 
Analogously to the method depicted before, here too, in step 
300, a function is read in that indicates the pattern of the 
lane marking (right and/or left side) , which was derived from 

15 the image data of the image sensor system. In subsequent step 
302, for first phase 4 (first time range, first distance 
range) one or more future tracks of the vehicle (e.g. track of 
the left vehicle edge and/or track of the right vehicle edge) 
are determined, for example, in the light of the vehicle speed 

20 and the steering angle and/or transverse acceleration, etc. In 
this phase, the assumption is made that there is no reaction 
by the driver, and consequently that no change in the steering 
angle or the transverse acceleration occurs. It has become 
apparent that a time range of 0.5 sec is suitable as the 

25 duration of this first phase. In this context, the track is 

also represented using a functional equation. In one exemplary 
embodiment, in step 302, based on the ascertained function, 
predefined points of the track of the vehicle, which, for 
example lie at predetermined distances, are determined, to 

3 0 which the vehicle will move towards the end of the first 

phase. Corresponding track points (at corresponding distances) 
have been ascertained with respect to the edge marking region. 
Then the second phase begins, for whose duration a time of 0.1 
sec has been proven to be suitable. In this phase the driver 

35 reacts by turning the steering wheel away from the side 

marking. In this context, it is assumed that, at the actual 
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time, a warning would have occurred. It is assumed that the 
steering wheel is turned by a certain value, or rather, that 
the vehicle travels a curve having a certain transverse 
acceleration. A 10° steering angle and 2 m/sec^ have shown 
5 themselves as suitable values. Consequently, in step 304, 

starting from the end point of the track of the first phase, a 
track of the vehicle for the second phase is determined, an 
appropriate correction of the steering angle and/or of the 
transverse acceleration being taken into consideration in the 

10 determination of the parameters of the function. After the end 
of the second phase, it is assumed that the vehicle will 
continue on at an unchanged steering angle, so that in one 
exemplary embodiment (not shown in Figure 4) an additional 
track calculation occurs. In the second phase also and 

15 possibly the third phase too, in one exemplary embodiment, 

individual track points are calculated at certain distances. 
After determination of the future track curve of the vehicle, 
checking is done in step 306 as to whether the function of the 
pattern of the lane marking has crossings with the track 

2 0 curves of the first or second phase. This is done by 

evaluating the function equations or by comparison of the 
coordinates of corresponding track points. If this is not the 
case, then it is assumed that a situation is not present in 
which the vehicle's leaving the lane is a threat. However, if 

25 a crossing is detected, then according to step 308 an optical 
and/or acoustical warning is given to the driver and/or a 
vehicle intervention for the automatic correction of the 
steering is ascertained and performed. The program is then 
ended and run through again at the next point in time . 

30 

Both in this method and the one depicted in the light of the 
first exemplary embodiment, the loudness of the warning, in 
the case of an acoustical warning, may be made variable and a 
function of the state of attentiveness of the driver. 



The effectiveness of the method shown in Figure 4 is made 
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clear in the light of the illustration in Figure 5. An x/y 
coordinate system is drawn in, whose centroid lies, for 
example in the middle of the vehicle. In this context, vehicle 
400 travels in a traffic lane which is characterized by edge 
5 markings 402 and 404. Two tracks of the vehicle are also drawn 
in, one left and one right (406, 408) which describe the 
course of the left and the right vehicles edges • As was 
pointed out above, the course of the left and the right side 
markings (402, 404) relative to the vehicle are ascertained by 

10 a camera system- The course of the track markings is then 

shown with the aid of a mathematical equation, e.g., a third 
order power function. In this manner, the side markings are 
able to be calculated, for instance by the five points A, B, 
C, D, E for the left side markings, and by the points F, G, H, 

15 J, K for the right side markings. These points are selected so 
that, on the assumption of constant travel speed of the 
vehicle, they lie at certain time intervals ahead of the 
vehicle (e.g. A, F at 0 sec, B, G at 0.25 sec, C, H at 0.5 
sec, D, J at 0.75 sec and E, K at 1 sec) . To determine the 

20 tracks of the vehicle, paths 406, 408 are ascertained for the 
left and the right track of the vehicle. In this context, the 
travel speed and the current steering angle and/or the 
transverse acceleration of the vehicle make a difference. It 
is assumed that the vehicle travels at least in the next 0.5 

25 sec using an unchanged steering angle (phase 1) . This 

corresponds to the area between points A and C, and F and H, 
respectively. The track of the vehicle is also represented by 
a mathematical equation corresponding to the representation of 
the pattern of the lane markings. In this manner, within the 

30 extent of the first phase, points L, M, N may be determined 
for the left track and Q, R, S for the right track. In this 
context, the equations are set up in such a manner that the 
left edge of the vehicle travels through the curve from L to 
N, and the right edge from Q to S. It is further assumed that 

35 the driver, if he were warned at the point in time shown, 

will, after ca 0.5 sec turn his steering wheel by a certain 
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magnitude (e.g. ca 10°) in the right direction, in order to 
avoid leaving the lane. In this manner, in phase 2 (area C to 
E and H to K, respectively, broken line area in phase 2) a new 
curve having a changed steering angle and/or a changed 
5 transverse acceleration is determined, which are determined by 
the points N, O and P and S, T and U, respectively. In this 
context, the track points of the vehicle tracks are calculated 
for the same distances as the track points of the lane 
markings, so that corresponding value pairs are formed. 

10 Thereafter, the patterns of the curves are compared (right 

track curve to right side edge curve, left track curve to left 
side marking curve) as to whether there are crossings or not. 
This occurs in the implementation shown by comparison of the y 
coordinate of corresponding track points. Thus, a warning on 

15 the left side occurs if either L lies above A, or M above B, 
or N above C, or O above D or P above E. Similarly, a warning 
on the right side occurs if either Q lies below F, or R below 
G, or S below H, or t below J or U below K. 

2 0 As an alternative to this, the warning occurs only if one of 

the points L to P lies above one of points A to E by a certain 
measure, as well as if one of points Q to U lies below one of 
points F to K by a certain measure. In this context, the value 
of the measure that was mentioned may also be negative. So 

25 that even an approach to the lane marking generates a warning. 
In this manner, tolerances are taken into consideration. 

In this context, depending on the exemplary embodiment, the 
warning may occur on both vehicle sides or only on one vehicle 
30 side. 

In the exemplary embodiment just shown, the future track of 
the vehicle is calculated on the basis of a supposedly fixed 
steering of the driver. In practice it turns out that, in the 
35 case of some drivers, relatively many false warnings may occur 
if such a driver swings back and forth between the lane 
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markings . 



Therefore, in an additional exemplary embodiment the warning 
threshold for triggering the lane- leaving warning is 
5 dynamically changed by taking into consideration the degree of 
attentiveness of the driver when the calculation is made of 
the future track of the vehicle . The calculation of the future 
track of the vehicle, and therewith the warning threshold for 
triggering the warning becomes, in this context, a function of 

10 the behavior (performance characteristics) of the driver in 
the recent past. Depending on how much the vehicle travels 
curved tracks due to the steering movements of the driver, the 
calculation of the future track of the vehicle, and thereby 
also the warning threshold itself are modified. In the case of 

15 attentive drivers, this avoids many false warnings, but in the 
case of inattentive drivers, the warnings are triggered at the 
right time. 

In general, in the above exemplary embodiment, a warning is 

2 0 triggered if a left warning boundary touches the left lane 

marking, or the right warning boundary touches the right lane 
marking- In this context, the warning boundary represents the 
future track of the vehicle, taking into consideration a 
possible estimated steering correction by the driver, if the 
25 latter drives closely towards the respective lane marking. The 
future track of the vehicle, and therewith the warning 
boundary are therefore curved. It has turned out that an 
inattentive driver, who is supposed to be warned by the 
system, as a rule makes very slight steering movements before 

3 0 leaving the lane. Therefore the vehicle travels a path having 

a relatively constant curvature. In the present exemplary 
embodiment, the warning boundaries are brought in with respect 
to each other if the curvature of the track that has been 
traveled by the vehicle has changed considerably (i.e. if 
35 comparatively strong steering movements have been executed by 
the driver) . This means that a greater curvature of the future 
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track of the vehicle away from the lane marking is assumed- In 
response to a relatively constant curvature of the traveled 
track (and thus at very slight steering movements of the 
driver) the warning boundaries are broadened out, i.e. the 
5 possible future track of the vehicle is calculated using a 
lower change in the curvature . 

In Figures 6 and 7 the vehicle is shown as 400. Furthermore, 
the left and right lane marking are shown as 402. 404 shows 

10 the future course of the vehicle without consideration of 
steering wheel changes, while the left and right warning 
boundaries are shown as 406 and 408, respectively. In this 
context, the left and right warning boundaries represent the 
future track of the vehicle while taking into consideration 

15 the correcting interventions of the driver. In this context, 
Figure 6 shows the situation in response to an attentive 
driver, while Figure 7 shows the situation in response to an 
inattentive driver. Consequently, in Figure 6 the curvature of 
the vehicle's course changed greatly in the past. It is for 

20 that reason that the warning boundaries run close together. In 
other words, the courses of the warning boundaries show a 
great deviation from the future course of the vehicle without 
steering correction. If the vehicle approaches the right or 
the left lane marking, a warning occurs only at a late time. 

25 In Figure 7 the situation is different. In the near past, the 
curvature of the course of the vehicle has hardly changed, and 
therefore the warning boundaries lie wide apart at a certain 
distance ahead of the vehicle. Thus, the courses of the 
warning boundaries here show a lesser deviation from the 

30 future course of the vehicle without steering correction. If 
the vehicle approaches closer to the lane marking, a warning 
is triggered in good time. 

In the exemplary embodiment, this behavior is implemented in 
35 that, with the aid of steering movements, recorded, for 

example, from the signal of a steering angle sensor or a yaw 
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rate sensor, a factor is determined using which the predefined 
correcting value is weighted. Consequently, when the 
calculation is made of the future track of the vehicle, and 
with that the course of the warning boundaries, a large 
5 correcting value is assumed in response to strong curvatures 

and steering movements, which leads to a greater change of the 
curvature of the future track, and with that, a drawing 
together of the warning boundaries, while in response to small 
steering movements in the past, a lower curvature factor is 
10 assumed, and consequently the warning boundaries lie far 
apart. In this context, the implementation occurs, as 
described in the case of the exemplary embodiments described 
above, within the scope of a computer program. 

15 The drawing together of the warning boundaries that was shown 
is not only undertaken as a function of the past of the 
vehicle's course, but, in one exemplary embodiment, 
alternatively or additionally even at time periods in which, 
for example, the measurement of the lane markings is 

20 uncertain. Therefore, in such situations a large factor is 
specified, which leads to a great curvature of the warning 
boundaries, and consequently to a drawing closer together of 
the warning boundaries. This avoids unnecessary signals as a 
response to uncertain recording of lane markings . 

25 In other exemplary embodiments, besides recognition of the 

attentiveness of the driver in the light of the curvature of 
the course of the vehicle in the past, or in the light of 
steering movements, a different recognition of attentiveness, 
such as one derived from accelerator movements, head posture, 

30 etc, is used. 
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